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Abstract 

Background: In today's dynamic health-care system, organizations such as hospitals are required to improve their 
performance for multiple stakeholders and deliver an integrated care that means to work effectively, be innovative 
and organize efficiently. Achieved goals and levels of quality can be successfully measured by a multidimensional 
approach like Balanced Scorecard (BSC). The aim of the study was to verify the opportunity to introduce BSC 
framework to measure performance in St. Anna University Hospital of Ferrara, applying it to the Clinical Laboratory 
Operative Unit in order to compare over time performance results and achievements of assigned targets. 

Methods: In the first experience with BSC we distinguished four perspectives, according to Kaplan and Norton, 
identified Key Performance Areas and Key Performance Indicators, set standards and weights for each objective, 
collected data for all indicators, recognized cause-and-effect relationships in a strategic map. One year later we 
proceeded with the next data collection and analysed the preservation of framework aptitude to measure 
Operative Unit performance. In addition, we verified the ability to underline links between strategic actions 
belonging to different perspectives in producing outcomes changes. 

Results: The BSC was found to be effective for underlining existing problems and identifying opportunities for 
improvements. The BSC also revealed the specific perspective contribution to overall performance enhancement. 
After time results comparison was possible depending on the selection of feasible and appropriate key 
performance indicators, which was occasionally limited by data collection problems. 

Conclusions: The first use of BSC to compare performance at Operative Unit level, in course of time, suggested 
this framework can be successfully adopted for results measuring and revealing effective health factors, allowing 
health-care quality improvements. 



Background 

Health-care organizations are operating in a complex 
environment. Financial pressures from government, the 
need to arrange integrated care and improve perfor- 
mance for multiple stakeholders, as well escalating costs 
are driving administrators to search for effective man- 
agement tools. In addition, all aspects of the sector are 
being asked to account for their performance and to 
demonstrate efficiency and effectiveness in providing 
services to their clients. 
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Financial measures alone are not sufficient to measure 
performance. Other factors missing from traditional finan- 
cial reporting such as competence, customer focus, opera- 
tional efficiency, innovation and knowledge must be 
carefully considered. Adopting Balanced Scorecard (BSC) 
in healthcare organization permits us to develop a more 
comprehensive set of performance indicators. The BSC is 
a management tool, originally applied to private sector, 
developed by Kaplan and Norton in 1992 [1]. Their frame- 
work broadened the traditional performance assessment 
approach by integrating financial measures with other key 
performance indicators linked to additional areas: custo- 
mer preferences, internal business processes, organization 
growth, learning and development. Performance measures 
belonging to all four features are included in BSC [2] . 



© 201 1 Lupi et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons 
BiolVlGCl CGITiTBI Attribution License ;http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in 
any medium, provided the original work is properly cited. 



Lupi ef al. Cost Effectiveness and Resource Allocation 201 1, 9:7 
http://www.resource-allocation.eom/content/9/1/7 



Page 2 of 5 



About ten years after Kaplan and Norton developed 
BSC, a number of health-care organizations started to 
adapt and implement this framework in various settings 
from North America to Asia [3-5] and also in Europe 
[6,7] with the remarkable experience of NHS Perfor- 
mance Assessment Framework [8] in United Kingdom. 
In the past few years a growing number of Italian health- 
care institutions adopted BSC with the aim of measuring 
overall performance and to improve clinical and financial 
goals [9]. 

When applied to the health-care sector, the four tradi- 
tional perspectives should be slightly modified to better 
display the functioning of public funded hospitals. The 
Financial Perspective should contain indicators of efficiency 
and asset utilization, including cost containment. Commu- 
nity Perspective should include measures of quality 
patient-centred care. Internal Processes Perspective should 
report indicators of continuous quality improvement and 
integrated service design. Growth and Learning Perspective 
should cover measures of human capital and strategic 
competencies. In each of the perspective significant success 
activities, indicated as Key Performance Areas (KPAs), are 
defined. Afterwards critical success factors, known as Key 
Performance Indicators (KPIs), are identified as well as 
measurement methods and standards. They balance 
between long term and short term in addition to internal 
and external factors contributing to business strategy that 
is translated into operational terms. Design of a strategic 
map, communicating outcomes to achieve by means of 
strategic initiatives for all Perspectives and their relation- 
ships, represents an essential component of BSC. 

Traditionally financial metrics obtain increased impor- 
tance than other parameters like quality of care, patient 
satisfaction, innovation, physicians and staff fulfillment. 

In consequence of Laboratory Analysis management 
and staff requests for being evaluated, not only for finan- 
cial outcomes, but also for relationships with community, 
internal procedures improvement, competence and 
knowledge, a first application of BSC was carried out 
with satisfactory results [10] in the past. In continuity 
with previous experience, the model was again applied, 
only with slight modifications to better depict Laboratory 
Analysis current activity. The objective of this paper is to 
confirm feasibility and value of using BSC to measure, 
over time, performance in Laboratory Analysis Operative 
Unit (OU) of St. Anna University Hospital, in particular 
the capacity to highlight outcome differences and explain 
their occurrence and relationships. 

Methods 

We followed the methodological procedure established 
for precedent performance measuring by BSC. Briefly, as 
previously described [10], the major steps were: 



• definition of strategic map divided into four Perspec- 
tives (Community, Internal Processes, Financial 
Resources, Growth and Learning) according Norton and 
Kaplan [1]; 

• identification of Key Performance Areas or macro- 
objectives, namely most important fields linked to 
abovementioned Perspectives in which not to fail [11]; 

• determination of cause/effect relationships between 
KPAs in order to explain interdependence among objec- 
tives belonging to different areas; 

• description of pre-defined sub-objectives OU have to 
realize in order to accomplish KPAs; 

• designation of Key Performance Indicators suitable 
for monitoring the degree of achievement of defined 
sub-objectives. In particular indicators that can effec- 
tively represent the phenomenon being measured were 
chosen from those reported in the text of Bocci and 
Miozzo [12] according to truth, focus, consistency, 
access, clarity, so what, timeliness, cost, gaming criteria 
as set by Neely and Kennerly [13]. 

• characterization of standard value (acceptable- 
expected value to obtain adequate quality of assistance) 
and weight (importance attributed to the indicator, high- 
est sum of weights of each Perspective was equal to 100). 
Standard values were established in agreement with 
health workers analysing previous experience of OU. 
Associated weights were set up on the basis of mean 
weight assigned in order to permit balanced evaluation of 
OU performance and emphasize key- objectives by a pool 
of professionals belonging to assistance, organisational 
and directional fields. The adopted system of allocation 
of weights allowed us to understand areas and indicators, 
among those included in the evaluation, assuming greater 
importance for the organization depending on the busi- 
ness strategy. 

• data collection; 

• data ordering in spreadsheets. 

Information has been drawn from various paper and 
digital sources. For example although St. Anna University 
Hospital does not have a computer platform dedicated to 
BSC, data is derived from SAP (cost containment, ticket 
collection), LIS (intra-lab reproducibility of results, activity 
indicators, external and internal TAT,) project SOLE data- 
base (number of GPs in the network, multi-typology of 
report receiving), quality indicators intranet database 
(MISA score), training office database (staff refresher 
courses). 

At the second survey strategic map was confirmed. 
Minor changes were operated when chosen indicators 
were no longer detectable or available because of the 
ongoing transition of Laboratory Analysis in the uni- 
fied Department with Ferrara Local Health Unit. In 
order to get an overall performance assessment for 
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each Perspective, the weights of achieved goals were 
added and the obtained value was translated in a pic- 
torial representation as traffic lights. Colour assigned 
in a summary table corresponding to prevalent assess- 
ment. Totally achieved goals are represented by green, 
completely not achieved goals are symbolized by red, 
while orange indicated a borderline condition due to 
an observed value slightly out of line with the fixed 
target or insurmountable difficulties which did not 
allow to reach the objective. 

The first data collection partly referred to 2007 and 
partly to January-June 2008 because some indicators 
related to activities implemented at the beginning of 
2008. The second data collection referred to the second 
part of 2008 and 2009. 

On the basis of obtained data, a general evaluation panel 
of the entire Operative Unit was built, summarizing, for 
each Perspective, the achieved level of performance. 

Results 

Data collection in two different surveys and Perspectives 
schedules completion permitted us to get an evaluation 
of performance trend over time. Results and a brief 
description of strategic map are shown below. 

Strategic map 

Strategic map was built on four classical perspectives 
identified by Kaplan and Norton [1] with the exception 
of Community Perspective that was defined as an area 
including objectives linked to different stakeholders: 
Users such as as patients, hospital doctors, general prac- 
titioners; Owners of public healthcare services; Public 
entities including laws that protect community and 
environment. Internal Procedures Perspective referred 
to how specific processes are performed, including ser- 
vice appropriateness and innovation, relationship with 
users, quality of hygiene and organisational standards, 
risk management, accreditation. To ensure financial sus- 
tainability was identified as main objective in Financial 
Resources Perspective, taking into account the urgent 
need to ensure the financial stability of public healthcare 
hospital. For Growth and Learning Perspective we con- 
sidered staff continuous updating, improvement of com- 
puter infrastructure supporting informative flows, 
organisational resources as team-work, leadership, align- 
ment to organisation strategy. Objectives have been 
identified trying to keep in mind the link between them 
and different perspectives. The map was drawn up in 
the belief that it can be a valuable tool allowing the 
reading of close cause-effect relationships between var- 
ious strategic objectives and enabling their accomplish- 
ment. Review at the second survey confirmed the 
structure previously outlined that is shown in Additional 
File 1. 



Community Perspective 

In this Perspective (Additional File 2), six of the 
assigned objectives were fully met (score 57 out 100), 
while two presented a borderline condition and two 
others showed a misalignment highlighted by red colour. 
Compared to first survey, some conditions have chan- 
ged: we reported an improvement in providing timely 
responses to emergency requests, moving from a critical 
situation to a value slightly below the standard set, but 
also a worsening due to increase of complaints. 

The overall Perspective assessment, with recorded 
improvements, described a positive trend therefore pic- 
torial representation has changed from orange to green. 

Internal Processes Perspective 

Almost all given objectives resulted in achieved levels 
and were therefore marked by a green signal (Additional 
File 3). The score was 65 out of 95 because a goal with 
weight 5 could not be evaluated and was postponed to 
next year. We showed a reduction in the average num- 
ber of withdrawals per operator and a trend to align- 
ment for optimization of urgent analysis procedures. 
Both conditions are marked with orange. General eva- 
luation was wholly satisfactory and then characterized 
by green colour confirming a positive result of the first 
survey. 

Financial Resources Perspective 

Objectives assessed were almost totally aligned with set 
standards (Additional File 4) with a score of 75 out of 
100. The global level of performance is therefore still 
marked, also for the second detection, with a green 
signal. 

Growth and Learning Perspective 

Allocated goals were entirely achieved reaching a score 
of 70 out of 70 (Additional File 5) because the objective 
related to operators' satisfaction and could not be evalu- 
ated since no further organisational wellbeing survey has 
been conducted. Positive green indication, as obtained in 
first survey, was confirmed. 

Overall evaluation 

Comparison between the two surveys, reported in sum- 
mary table (Additional File 6), confirmed a fully satisfac- 
tory performance highlighted by green colour for all 
considered aspects. Particularly overtime evaluation 
pointed out improvement reached in Community 
Perspective. 

Discussion 

Application of experimental model allowed to build a 
summary table showing changes in performance according 
to selective Perspectives. In the first survey, Community 
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Perspective obtained an orange alert signal indicating a 
situation of partial mismatch for some of its objectives, as 
other Perspectives showed a complete achievement of 
fixed goals. While Internal Processes, Financial Resources 
and Growth and Learning Perspectives resulted unmodi- 
fied, confirming a positive performance, some improve- 
ments have changed Community Perspective situation 
leading to a positive signal in the second survey. 

Reasons that led to this final score can be better high- 
lighted considering in detail specific tables. Maintaining 
the positive performance was not simply due to a stable 
condition but several improvements were noticed. Particu- 
larly, for Community Perspective, we found enhanced abil- 
ity to provide a timely response to emergency requests, an 
increased number of reports delivered by web, reduction 
of waiting times for access to appointments. However, 
there are also some deteriorations, such as a lack of identi- 
fication badge for staff in contact with patients and the ris- 
ing of complaints (mainly due to changes in set of rules for 
payment). Also other Perspectives confirmed positive 
results due to the maintenance of capacity to achieve the 
objectives and further improvement efforts, including bet- 
ter analytical accuracy, increased number of GPs becom- 
ing web connected, largest commitment of staff in 
teaching activities and tutoring. 

The tested and adopted BSC model proved to be use- 
ful in highlighting variations by changing colours in pic- 
torial representation for Community Perspective. 
Actually this Perspective experienced greatest ferment 
situation characterized by improvements and worsening. 
Furthermore, the model allowed us to get critical issues 
of an indicator, its impact on the area to which it 
belongs, allowing analysis, maintaining the indicator 
controlled and preparing corrective actions. 

Applying our experimental model of BSC, we were 
interested to test the understanding of interdependence 
relations between the different Perspectives to confirm 
the assumptions on which construction of Strategic 
Map, indicative of the strategy is based [14,15]. In detail, 
as showed in Additional File 1 and illustrated by blue 
arrows, the improvement in information technology in 
Growth and Learning Perspective, with increased num- 
ber of web connected GPs, has led to an augmented 
number of reports delivered by web. This issue was 
included in innovation in service production and offer, a 
KPA of Internal Processes Perspective that can be con- 
nected to service appropriateness and meeting the 
health needs of the population in collaboration with 
other operators, KPAs of Community Perspective. It is 
reasonable to assume an upcoming positive influence on 
business synergies and economic efficiency, KPAs 
belonging to Financial Resources Perspective. 

In reference to the two observed worsening situations, 
they were not shown by pictorial representation 



probably because these indicators have been assigned a 
low weight. A further explanation could be related to 
the type of reporting chosen, that is based on three 
levels, while adoption of a system more appropriate to 
emphasize borderline situations could solve this 
problem. 

Conclusions 

As exposed in previous work [10], the experienced BSC 
model showed strengths and weaknesses, however it was 
found to be effective for underlining existing problems 
and identifying opportunities for improvement, as con- 
firmed in this paper. In addition we assessed the ability 
to capture connections of measured results to strategy 
and their cause-and-effect linkages that describe the 
hypotheses of the strategy [16]. Main difficulties lie in 
choosing appropriate indicators and the subsequent 
assignment of weights, avoiding under or over estima- 
tion, and standards. Preference must be calibrated 
according to specific situation and must not allow gen- 
eralizations to better describe context of reference. 

Furthermore performance comparisons using the BSC 
depend on selection of feasible and appropriate Key Per- 
formance Indicators, which is occasionally limited by data 
collection problems, for example, constant updating to 
adapt to evolving context changes may impose KPI varia- 
tions, inducing the lack of reference to historical data. 

BSC was an ideal point of contact between clinical and 
economic dimension and allowed us to perceive 
improved results as a consequence of progress in differ- 
ent inter-related perspectives. 

Additional material 



Additional file 1: Strategic Map_Additional file 1 The file contains a 
strategic map of Laboratory Analysis in which we highlighted links 
between KPIs of different KPAs, as emerged from results of the two 
surveys, that have led to an improvement found in the overall 
performance of Operative Unit. 

Additional file 2: Community Perspective Table_Additional file 2 

The file contains a table resuming macro- and specific objectives 
referring to KPAs, indicators and standards referring to KPIs, results 
obtained in the two different observations of Community Perspective. 

Additional file 3: Internal Processes Perspective Table_ Additional 
file 3. The file contains a table resuming macro- and specific objectives 
referring to KPAs, indicators and standards referring to KPIs, results 
obtained in the two different observations of Internal Processes 
Perspective. 

Additional file 4: Financial Resources Perspective Table Additional 
file 4. The file contains a table resuming macro- and specific objectives 
referring to KPAs, indicators and standards referring to KPIs, results 
obtained in the two different observations of Financial Resources 
Perspective. 

Additional file 5: Growth and Learning Perspective Table Additional 
file 5. The file contains a table resuming macro- and specific objectives 
referring to KPAs, indicators and standards referring to KPIs, results 
obtained in the two different observations of Growth and Learning 
Perspective. 
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Additional file 6: Global Performance Table_Additional file 6 The file 
contains a table resuming global performance reached in all four 
Perspectives. 



Abbreviations 

Balanced Scorecard: BSC; Coefficient of Variation: CV; General Practitioner: GP; 
Key Performance Area: KPA; Key Performance Indicator: KPI; Laboratory 
Information System: LIS; Mean Index of Deviation: MISA; Mixed Advisory 
Committee: MAC; Operative Unit: OU; Oral Anticoagulant Therapy: OAT; 
Sanita On LinE (e-Health): SOLE; Time Around Time: TAT; Unique 
Appointment Centre: UAC. 
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